Differentiating poor validity from probable impairment on the medical symptom validity test: a cross-validation study.

AIMS: In neuropsychological evaluations, it is often difficult to ascertain whether poor performance on measures of validity is due to poor effort or malingering, or whether there is genuine cognitive impairment. Dunham and Denney created an algorithm to assess this question using the Medical Symptom Validity Test (MSVT). We assessed the ability of their algorithm to detect poor validity versus probable impairment, and concordance of failure on the MSVT with other freestanding tests of performance validity. METHODS: Two previously published datasets (n = 153 and n = 641, respectively) from outpatient neuropsychological evaluations were used to test Dunham and Denney's algorithm, and to assess concordance of failure rates with the Test of Memory Malingering and the forced choice measure of the California Verbal Learning Test, two commonly used performance validity tests. RESULTS: In both datasets, none of the four cutoff scores for failure on the MSVT (70%, 75%, 80%, or 85%) identified a poor validity group with proportionally aligned failure rates on other freestanding measures of performance validity. Additionally, the protocols with probable impairment did not differ from those with poor validity on cognitive measures. CONCLUSIONS: Despite what appeared to be a promising approach to evaluating failure on the easy MSVT subtests when clinical data are unavailable (as recommended in the advanced interpretation program, or advanced interpretation [AI], of the MSVT), the current findings indicate the AI remains the gold standard for doing so. Future research should build on this effort to address shortcomings in measures of effort in neuropsychological evaluations.

Use of Latent Class Analysis to define groups based on validity, cognition, and emotional functioning.

OBJECTIVE: Latent Class Analysis (LCA) was used to classify a heterogeneous sample of neuropsychology data. In particular, we used measures of performance validity, symptom validity, cognition, and emotional functioning to assess and describe latent groups of functioning in these areas. METHOD: A data-set of 680 neuropsychological evaluation protocols was analyzed using a LCA. Data were collected from evaluations performed for clinical purposes at an urban medical center. RESULTS: A four-class model emerged as the best fitting model of latent classes. The resulting classes were distinct based on measures of performance validity and symptom validity. Class A performed poorly on both performance and symptom validity measures. Class B had intact performance validity and heightened symptom reporting. The remaining two Classes performed adequately on both performance and symptom validity measures, differing only in cognitive and emotional functioning. In general, performance invalidity was associated with worse cognitive performance, while symptom invalidity was associated with elevated emotional distress. CONCLUSIONS: LCA appears useful in identifying groups within a heterogeneous sample with distinct performance patterns. Further, the orthogonal nature of performance and symptom validities is supported.

Clinimetric validity of the Trail Making Test Czech version in Parkinson's disease and normative data for older adults.

OBJECTIVE: The influence of demographic variables on the Trail Making Test (TMT) performance in older individuals and empirical findings on clinical validity in predementia states, such as Parkinson's disease mild cognitive impairment (PD-MCI), are limited. The principal aim of this study was to add normative data for the Czech population of older adults and explore the clinimetric properties between PD-MCI and PD patients with normal cognition (PD-NC). METHOD: The study included 125 PD patients classified as 77 PD-MCI and 48 PD-NC and 528 older individuals (60-74 years, further subdivided for normative tables into 60-64, 65-69 and 70-74 age groups) and very old individuals (aged 75-96, further subdivided into 75-79, 80-84, 85-96) cognitively intact Czech adults. RESULTS: Mostly age, to a lesser extent education but not gender, was associated with most TMT basic and derived indices (TMT-B - A). However, the ratio of TMT-B/TMT-A was independent of both age and education. We provide corresponding T-scores that minimize the effect of demographic variables. The results showed a high discriminative validity of TMT basic and derived indices for the differentiation of PD-MCI from PD-NC (all p < .05). The classification accuracy for the differentiation of PD-MCI from controls was optimal for the TMT-B only (80% area under the curve) based on norm adjusted scores. The classification accuracy of the TMT for PD-MCI vs. PD-NC was suboptimal. CONCLUSIONS: The cut-offs and normative standards are useful in clinical practice for those working with PD patients and very old adults.

Strategies of successful and unsuccessful simulators coached to feign traumatic brain injury.

OBJECTIVE: The present study evaluated strategies used by healthy adults coached to simulate traumatic brain injury (TBI) during neuropsychological evaluation. METHOD: Healthy adults (n = 58) were coached to simulate TBI while completing a test battery consisting of multiple performance validity tests (PVTs), neuropsychological tests, a self-report scale of functional independence, and a debriefing survey about strategies used to feign TBI. RESULTS: "Successful" simulators (n = 16) were classified as participants who failed 0 or 1 PVT and also scored as impaired on one or more neuropsychological index. "Unsuccessful" simulators (n = 42) failed ≥2 PVTs or passed PVTs but did not score impaired on any neuropsychological index. Compared to unsuccessful simulators, successful simulators had significantly more years of education, higher estimated IQ, and were more likely to use information provided about TBI to employ a systematic pattern of performance that targeted specific tests rather than performing poorly across the entire test battery. CONCLUSION: Results contribute to a limited body of research investigating strategies utilized by individuals instructed to feign neurocognitive impairment. Findings signal the importance of developing additional embedded PVTs within standard cognitive tests to assess performance validity throughout a neuropsychological assessment. Future research should consider specifically targeting embedded measures in visual tests sensitive to slowed responding (e.g. response time).

Evaluating the Medical Symptom Validity Test (MSVT) in a Sample of Veterans Between the Ages of 18 to 64.

The purpose of the current study was to compare three potential profiles of the Medical Symptom Validity Test (MSVT; Pass, Genuine Memory Impairment Profile [GMIP], and Fail) on other freestanding and embedded performance validity tests (PVTs). Notably, a quantitatively computed version of the GMIP was utilized in this investigation. Data obtained from veterans referred for a neuropsychological evaluation in a metropolitan Veteran Affairs medical center were included (N = 494). Individuals age 65 and older were not included to exclude individuals with dementia from this investigation. The sample revealed 222 (45%) in the Pass group. Of the 272 who failed the easy subtests of the MSVT, 221 (81%) met quantitative criteria for the GMIP and 51 (19%) were classified as Fail. The Pass group failed fewer freestanding and embedded PVTs and obtained higher raw scores on all PVTs than both GMIP and Fail groups. The differences in performances of the GMIP and Fail groups were minimal. Specifically, GMIP protocols failed fewer freestanding PVTs than the Fail group; failure on embedded PVTs did not differ between GMIP and Fail. The MSVT GMIP incorporates the presence of clinical correlates of disability to assist with this distinction, but future research should consider performances on other freestanding measures of performance validity to differentiate cognitive impairment from invalidity.

Neuropsychological test validity in Veterans presenting with subjective complaints of 'very severe' cognitive symptoms following mild traumatic brain injury.

OBJECTIVE: This study explored the utility of combining data from measures of performance validity and symptom validity among Veterans undergoing neuropsychological evaluation for mild traumatic brain injury (mTBI). BACKGROUND: Persistent cognitive impairments following mTBI are often reported by returning combat veterans. However, objectively-measured cognitive deficits are not common among individuals with mTBI, raising the question of whether negative impression management influences self-ratings. METHODS: Self-report ratings were obtained for memory, concentration, decision-making, and processing speed/organization using a 5-point scale ranging from 'none' to 'very severe'. Veterans also completed brief neuropsychological testing which included measures of performance validity. RESULTS: Study 1 examined data from 122 participants and demonstrated that veterans reporting a 'very severe' cognitive deficit were over three times as likely to demonstrate poor effort on a validity test than those without a very severe rating. Study 2 replicated these findings in an independent sample of 127 veterans and also demonstrated that both severity of self-report ratings and performance on an embedded measure of effort were predictive of poor effort on a stand-alone performance validity test. CONCLUSION: Veterans with suspected mTBI who report 'very severe' cognitive impairment have a greater likelihood of putting forth sub-optimal effort on objective testing.

The Mild Brain Injury Atypical Symptoms (mBIAS) scale in a mixed clinical sample.

INTRODUCTION: The Mild Brain Injury Atypical Symptoms (mBIAS) scale was developed as a symptom validity test (SVT) for use with patients following mild traumatic brain injury. This study was the first to examine the clinical utility of the mBIAS in a mixed clinical sample presenting to a Department of Veterans Affairs (VA) neuropsychology clinic. METHOD: Participants were 117 patients with mixed etiologies (85.5% male; age: M = 39.2 years, SD = 11.6) from a VA neuropsychology clinic. Participants were divided into pass/fail groups using two different SVT criteria, based on select validity scales from the Minnesota Multiphasic Personality Inventory-2 (MMPI-2): first, Infrequency Scale (F) scores: (a) MMPI-F-Fail (n = 21) and (b) MMPI-F-Pass (n = 96); and, second, Symptom Validity Scale (FBS) scores: (a) MMPI-FBS-Fail (n = 36) and (b) MMPI-FBS-Pass (n = 81). RESULTS: The mBIAS demonstrated good internal consistency, and each item contributed meaningfully to the total score. At a symptom exaggeration base rate of 35%, an mBIAS cutoff of ≥11 was optimal for screening symptom exaggeration when groups were classified using both F and FBS scales. This cutoff score resulted in very high specificity (.89 to .94); moderate-high positive predictive power (.71 to .75) and negative predictive power (.72 to .79); and low-moderate sensitivity (.31 to .57). At all base rates of probable somatic exaggeration, a cutoff of ≥16 resulted in perfect specificity and positive predictive power, but very low sensitivity. CONCLUSIONS: The mBIAS has potential for use in samples outside of mild traumatic brain injury. In settings where the symptom exaggeration base rate is 35%, a cutoff of ≥11 may be used as a "red flag" for further evaluation, but should not be relied on for clinical decision making. At all base rates of probable somatic exaggeration, psychologists with patients who score ≥16 can be confident that those patients were exaggerating. Importantly, however, this cutoff may fail to identify a large proportion of patients who are exaggerating.

Embedded Measures of Performance Validity in the Rey Complex Figure Test in a Clinical Sample of Veterans.

The purpose of this study was to determine how well scores from the Rey Complex Figure Test (RCFT) could serve as embedded measures of performance validity in a large, heterogeneous clinical sample at an urban-based Veterans' Affairs hospital. Participants were divided into credible performance (n = 244) and noncredible performance (n = 87) groups based on common performance validity tests during their respective clinical evaluations. We evaluated how well preselected RCFT scores could discriminate between the 2 groups using cut scores from single indexes as well as multivariate logistic regression prediction models. Additionally, we evaluated how well memory error patterns (MEPs) could discriminate between the 2 groups. Optimal discrimination occurred when indexes from the Copy and Recognition trials were simultaneous predictors in logistic regression models, with 91% specificity and at least 53% sensitivity. Logistic regression yielded superior discrimination compared with individual indexes and compared with the use of MEPs. Specific scores on the RCFT, including the Copy and Recognition trials, can serve as adequate indexes of performance validity, when using both cut scores and logistic regression prediction models. We provide logistic regression equations that can be applied in similar clinical settings to assist in determining performance validity.

The Prague Stroop Test: Normative standards in older Czech adults and discriminative validity for mild cognitive impairment in Parkinson's disease.

OBJECTIVE: The aim of this study was to provide normative data for older and very old Czech adults on the Prague Stroop Test (PST) and to test its discriminative validity in individuals with Parkinson's disease mild cognitive impairment (PD-MCI). METHOD: The construction of the PST was modeled after the Victoria Stroop Test. We examined 539 participants aged 60-96 that met strict inclusion criteria. After, we compared the PST scores for a group of 45 PD-MCI patients with a healthy adult sample (HAS) of 45 age- and education-matched individuals. RESULTS: I. In the non-clinical sample, robust age- and education-related influences were observed on all PST scores. No gender effect was noted. II. For clinical cases, interference condition (PST-C) was able to discriminate between PD-MCI and HAS (all scores ps < .01). Area under the curve (AUC) was 77% when a screening cut-off of ≤ 27 s was used, showing sensitivity of 82% and specificity of 53%. A more conservative diagnostic cut-off of ≤ 33 s showed sensitivity of 60% and specificity of 80%. DISCUSSION: The present study provides PST normative data for basic, interference, and error scores stratified by age (60-96 years). PST appears to be a helpful tool for the diagnostics of PD-MCI especially in research settings at Level II (Litvan et al., 2012) and for PD-MCI attention/working memory and executive function subtyping.

Is Co-norming Required?

Researchers who have been responsible for developing test batteries have argued that competent practice requires the use of a "fixed battery" that is co-normed. We tested this assumption with three normative systems: co-normed, meta-regressed norms and a system of these two methods. We analyzed two samples: 330 referred patients and 99 undergraduate volunteers. The T scores generated for referred patients using the three systems were highly associated with one another and quite similar in magnitude, with an Overall Test Battery Means (OTBMs) using the co-normed, hybrid, and meta-regressed scores equaled 43.8, 45.0, and 43.9, respectively. For volunteers, the OTBMs equaled 47.4, 47.5, and 47.1, respectively. The correlations amongst these OTBMs across systems were all above .90. Differences among OTBMs across normative systems were small and not clinically meaningful. We conclude that co-norming for competent clinical practice is not necessary.

Wechsler Adult Intelligence Scale-IV Dyads for Estimating Global Intelligence.

All possible two-subtest combinations of the core Wechsler Adult Intelligence Scale-IV (WAIS-IV) subtests were evaluated as possible viable short forms for estimating full-scale IQ (FSIQ). Validity of the dyads was evaluated relative to FSIQ in a large clinical sample (N = 482) referred for neuropsychological assessment. Sample validity measures included correlations, mean discrepancies, and levels of agreement between dyad estimates and FSIQ scores. In addition, reliability and validity coefficients were derived from WAIS-IV standardization data. The Coding + Information dyad had the strongest combination of reliability and validity data. However, several other dyads yielded comparable psychometric performance, albeit with some variability in their particular strengths. We also observed heterogeneity between validity coefficients from the clinical and standardization-based estimates for several dyads. Thus, readers are encouraged to also consider the individual psychometric attributes, their clinical or research goals, and client or sample characteristics when selecting among the dyadic short forms.

Embedded measures of performance validity using verbal fluency tests in a clinical sample.

The objective of this study was to determine to what extent verbal fluency measures can be used as performance validity indicators during neuropsychological evaluation. Participants were clinically referred for neuropsychological evaluation in an urban-based Veteran's Affairs hospital. Participants were placed into 2 groups based on their objectively evaluated effort on performance validity tests (PVTs). Individuals who exhibited credible performance (n = 431) failed 0 PVTs, and those with poor effort (n = 192) failed 2 or more PVTs. All participants completed the Controlled Oral Word Association Test (COWAT) and Animals verbal fluency measures. We evaluated how well verbal fluency scores could discriminate between the 2 groups. Raw scores and T scores for Animals discriminated between the credible performance and poor-effort groups with 90% specificity and greater than 40% sensitivity. COWAT scores had lower sensitivity for detecting poor effort. A combination of FAS and Animals scores into logistic regression models yielded acceptable group classification, with 90% specificity and greater than 44% sensitivity. Verbal fluency measures can yield adequate detection of poor effort during neuropsychological evaluation. We provide suggested cut points and logistic regression models for predicting the probability of poor effort in our clinical setting and offer suggested cutoff scores to optimize sensitivity and specificity.

Utility of the Montreal Cognitive Assessment and Mini-Mental State Examination in predicting general intellectual abilities.

OBJECTIVE: To determine whether scores from 2 commonly used cognitive screening tests can help predict general intellectual functioning in older adults. BACKGROUND: Cutoff scores for determining cognitive impairment have been validated for both the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE). However, less is known about how the 2 measures relate to general intellectual functioning as measured by the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV). METHODS: A sample of 186 older adults referred for neuropsychological assessment completed the MoCA, MMSE, and WAIS-IV. Regression equations determined how accurately the screening measures could predict the WAIS-IV Full Scale Intelligence Quotient (FSIQ). We also determined how predictive the MoCA and MMSE were when combined with 2 premorbid estimates of FSIQ: the Test of Premorbid Functioning (TOPF) (a reading test of phonetically irregular words) and a predicted TOPF score based on demographic variables. RESULTS: MoCA and MMSE both correlated moderately with WAIS-IV FSIQ. Hierarchical regression models containing the MoCA or MMSE combined with TOPF scores accounted for 58% and 49%, respectively, of the variance in obtained FSIQ. Both regression equations accurately estimated FSIQ to within 10 points in >75% of the sample. CONCLUSIONS: Both the MoCA and MMSE provide reasonable estimates of FSIQ. Prediction improves when these measures are combined with other estimates of FSIQ. We provide 4 equations designed to help clinicians interpret these screening measures.

Comparisons of five performance validity indices in bona fide and simulated traumatic brain injury.

A number of performance validity tests (PVTs) are used to assess memory complaints associated with traumatic brain injury (TBI); however, few studies examine the concordance and predictive accuracy of multiple PVTs, specifically in the context of combined models in known-group designs. The present study compared five widely used PVTs: the Test of Memory Malingering (TOMM), Medical Symptom Validity Test (MSVT), Reliable Digit Span (RDS), Word Choice Test (WCT), and California Verbal Learning Test - Forced Choice (CVLT-FC). Participants were 51 adults with bona fide moderate to severe TBI and 58 demographically comparable healthy adults coached to simulate memory impairment. Classification accuracy of individual PVTs was evaluated using logistic regression and receiver operating characteristic (ROC) curves, examining both the dichotomous cutting scores as recommended by the test publishers and continuous scores for the measures. Results demonstrated nearly equivalent discrimination ability of the TOMM, MSVT, and CVLT-FC as individual predictors, all of which markedly outperformed the WCT and RDS. Models of combined PVTs were examined using Bayesian information criterion statistics, with results demonstrating that diagnostic accuracy showed only small to modest growth when the number of tests was increased beyond two. Considering the clinical and pragmatic issues in deriving a parsimonious assessment battery, these findings suggest that using the TOMM and CVLT in conjunction or the MSVT and CVLT in conjunction maximized predictive accuracy as compared to a single index or an assortment of these widely used measures.

Finger Tapping Test performance as a measure of performance validity.

The Finger Tapping Test (FTT) has been presented as an embedded measure of performance validity in most standard neuropsychological evaluations. The present study evaluated the utility of three different scoring systems intended to detect invalid performance based on FTT. The scoring systems were evaluated in neuropsychology cases from clinical and independent practices, in which credible performance was determined based on passing all performance validity measures or failing two or more validity indices. Each FTT scoring method presented with specificity rates at approximately 90% and sensitivity of slightly more than 40%. When suboptimal performance was based on the failure of any of the three scoring methods, specificity was unchanged and sensitivity improved to 50%. The results are discussed in terms of the utility of combining multiple scoring measures for the same test as well as benefits of embedded measures administered over the duration of the evaluation.

Czech version of Rey Auditory Verbal Learning test: normative data.

The present study provides normative data stratified by age for the Rey Auditory Verbal Learning test Czech version (RAVLT) derived from a sample of 306 cognitively normal subjects (20-85 years). Participants met strict inclusion criteria (absence of any active or past neurological or psychiatric disorder) and performed within normal limits on other neuropsychological measures. Our analyses revealed significant relationships between most RAVLT indices and age and education. Normative data are provided not only for basic RAVLT scores, but for the first time also for a variety of derived (gained/lost access, primacy/recency effect) and error scores. The study confirmed a logarithmic character of the learning slope and is consistent with other studies. It enables the clinician to evaluate more precisely subject's RAVLT memory performance on a vast number of indices and can be viewed as a concrete example of Quantified Process Approach to neuropsychological assessment.

WAIS-IV reliable digit span is no more accurate than age corrected scaled score as an indicator of invalid performance in a veteran sample undergoing evaluation for mTBI.

Reliable Digit Span (RDS) is a measure of effort derived from the Digit Span subtest of the Wechsler intelligence scales. Some authors have suggested that the age-corrected scaled score provides a more accurate measure of effort than RDS. This study examined the relative diagnostic accuracy of the traditional RDS, an extended RDS including the new Sequencing task from the Wechsler Adult Intelligence Scale-IV, and the age-corrected scaled score, relative to performance validity as determined by the Test of Memory Malingering. Data were collected from 138 Veterans seen in a traumatic brain injury clinic. The traditional RDS (≤ 7), revised RDS (≤ 11), and Digit Span age-corrected scaled score ( ≤ 6) had respective sensitivities of 39%, 39%, and 33%, and respective specificities of 82%, 89%, and 91%. Of these indices, revised RDS and the Digit Span age-corrected scaled score provide the most accurate measure of performance validity among the three measures.

Assessing effort: differentiating performance and symptom validity.

The current study aimed to clarify the relationship among the constructs involved in neuropsychological assessment, including cognitive performance, symptom self-report, performance validity, and symptom validity. Participants consisted of 120 consecutively evaluated individuals from a veteran's hospital with mixed referral sources. Measures included the Wechsler Adult Intelligence Scale-Fourth Edition Full Scale IQ (WAIS-IV FSIQ), California Verbal Learning Test-Second Edition (CVLT-II), Trail Making Test Part B (TMT-B), Test of Memory Malingering (TOMM), Medical Symptom Validity Test (MSVT), WAIS-IV Reliable Digit Span (RDS), Post-traumatic Check List-Military Version (PCL-M), MMPI-2 F scale, MMPI-2 Symptom Validity Scale (FBS), MMPI-2 Response Bias Scale (RBS), and the Postconcussive Symptom Questionnaire (PCSQ). Six different models were tested using confirmatory factor analysis (CFA) to determine the factor model describing the relationships between cognitive performance, symptom self-report, performance validity, and symptom validity. The strongest and most parsimonious model was a three-factor model in which cognitive performance, performance validity, and self-reported symptoms (including both standard and symptom validity measures) were separate factors. The findings suggest failure in one validity domain does not necessarily invalidate the other domain. Thus, performance validity and symptom validity should be evaluated separately.

Number of impaired scores as a performance validity indicator.

This study examined embedded performance validity indicators (PVI) based on the number of impaired scores in an evaluation and the overall test battery mean (OTBM). Adult participants (N = 175) reporting traumatic brain injury were grouped using eight PVI. Participants who passed all PVI (n = 67) demonstrated fewer impaired scores and higher OTBM than those who failed two or more PVI (n = 66). Impairment was defined at three levels: T scores < 40, 35, and 30. With specificity ≥.90, sensitivity ranged from .51 to .71 for number of impaired scores and .74 for OTBM.